About

Arun Nair is a highly skilled Data Engineer with over 15 years of experience in the field. He is currently working as a Data Engineer (Big Data/AWS) at AVY Consulting Limited, where he is responsible for working with AWS, Athena, S3, Glue, Kafka, Confluent Kafka connectors & Sinks, Apache Airflow, Spark, Java 8, CockroachDB, Helm, Python, Kubernetes, Docker, and Gradle. Prior to this, he worked with Lloyds Banking Group as a Big Data Engineer, where he was responsible for designing and building data processing pipelines using tools and frameworks in the Hadoop ecosystem to ingest structured and unstructured data. He also implemented and configured big data technologies as well as fine-tuned Spark component to handle high data traffic in millions. Arun has also worked with UK Home Office as a Big Data Engineer, where he developed an ingestion pipeline using Apache NIFI and Kafka into Graph DB (Janu graph) with HBase and Elasticsearch for back store and index (Write Path). On the Read path, he exposed Gremlin/Tinkerpop queries via REST endpoints. He also created a graph schema generation tool for data model. Arun holds a Bachelor's degree in Computer Application Development from NIIT, Bangalore, India. He has also received training from IBM. Arun has a wealth of experience in AWS, Data Engineering, Big Data, Spark, Kafka, Apache Airflow, Amazon Athena, Hadoop, Kubernetes, Java, Python, ETL, HDFS, Docker, Data Scientist, Research Scientist, Elastic Search, Apache Hive, and Search. He is a highly motivated, detail-oriented, and result-driven individual who is passionate about solving complex problems and delivering high-quality solutions.